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Abstract 

A fundamental question of human society is the evolution of cooperation. 
Many previous studies explored this question via setting spatial background, 
where players obtain their payoffs by playing game with their nearest neigh- 
bors. Another undoubted fact is that environment plays an important role 
in the individual development. Inspired by these phenomena, we reconsider 
the definition of individual fitness which integrates the environment, denoted 
by the average payoff of all individual neighbors, with the traditional indi- 
vidual payoffs by introducing a selection parameter u. Tuning u equal to 
zero returns the traditional version, while increasing u bears the influence 
of environment. We find that considering the environment, i.e., integrat- 
ing neighborhoods in the evaluation of fitness, promotes cooperation. If we 
enhance the value of u, the invasion of defection could be resisted better. 
We also provide quantitative explanations and complete phase diagrams pre- 
senting the influence of environment on the evolution of cooperation. Finally, 
the universality of this mechanism is testified for different neighborhood sizes, 
different topology structures and different game models. Our work may shed 
a light on the emergence and persistence of cooperation in our life. 
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1. Introduction 



The evolution of cooperation among unrelated individuals has become 
a major challenge in the biology and evolution research since the altruism 
and unselfish seems incompatible with Darwinian selection. Nevertheless, 
cooperation can be abundantly found in animal and human societies. In 
order to interpret these universal cooperative phenomena, evolutionary game 
theory has become a useful tool to study this puzzling dilemma y, [sj. 
Of particular renown is the prisoner's dilemma game, which as a paradigm 
illustrates the social confiict between altruistic and selfish behaviors and has 
attracted much attention in both theoretical and experimental studies j^. 
In the original form of the game, there exist two players, and they have to 
simultaneously decide whether to cooperate (C) or to defect (D). Naturally, 
their payoffs depend on their decisions: they can get R (P) while mutual 
cooperation (defection); if a defector meets a cooperator, the former can 
obtain a maximum individual payoff T and the latter can only get a minimal 
payoff S. The ranking of the four payoffs must satisfy T > R > P > S 
and 2R > T + S. Since the defector always outperforms the cooperator, 
irrespective of the opponent's choice, two players will inevitably fall into 
the mutual defection state. Over the past decades, many mechanisms have 
been proposed to overcome this dilemma, such as kin selection jil, direct and 
indirect reciprocity (g], 0], voluntary participation jsj and spatial structure 
[ol, [lo[. Among these achievements, the most prominent success is obtained 
by the spatial extension. 



In the pioneer work by Nowak and May [10|, players were arranged on 
the vertex of a square lattice, and their payoffs were gathered from playing 
the game with their nearest neighbors. Then players were allowed to adopt 
the strategy of their neighbors, provided their payoff was higher. It was 
shown that the emergence and sustainment of cooperation could be greatly 
improved. Along this way, much effort has been given to discover new mech- 
anisms that can sustain stable cooperation. They include the interplay be- 



tween evolutionary games and network structure (regular networks ll|, |l2 



small world networ ks [lil. 



ronmental noise 19, 20, 21 



14j . Il5| and scale-free networks [16|, ll7|, Il8|), e nvi 



additional noise introducing into payoffs [22|, |23 



teaching activity |24j . l25l |26| , the mobility of plaver s |27l . |28| , asymmetric pay 



off 129|, 

3; 



32 
34 



30l |. differences in evolutionary time scales [SVl, aspiring to the fittest 
For recent surveys of this field, one can refer to two extensive reviews 
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In the present work, we mainly focus on the research of individual fitness 
during the process of evolution. For the majority of previous researches, a 
player's fitness in the spatial structure equals the current accumulated payoff 
collected from its direct neighbors. Indeed, the meaning of fitness is often 



related to the reproduction ability [36|, |37| and some works exhibiting the 



relationships between payoff and fitness are also presented. For example, 
Ohtsuki et.al. defined fitness as the accumulated payoff with a background 
noise fssl; Szolnoki et.al. featured fitness as mixture of accumulated payoff 
and normalized payoff Jia et.al. drew fitness as the accumulated payoff 



with a fluctuation coefficient 401; and other researchers described fitness as 



combination of current payoff and past payoffs [4l|, |42|, |43| . Inspired by their 



instructive suggestions, we propose a new method to define individual fitness 
where neighborhoods are considered. 

As is known, environment plays an indispensable role in the real life and 
it can usually affect individual development. For example, people in the elite 
teams are more likely to learn technical ability from their excellent teammates 
to enhance competitiveness of themselves, animals in the strong groups are 
more easily to prey or resist the predators. Therefore, it is natural to take 
environment presented by the average payoff of all individual neighbors into 
account when talking about individual fitness. Here we redefine the individ- 
ual fitness via integrating its current accumulated payoff with average payoff 
of all individual neighbors, and find that the evolution of cooperation can be 
remarkably promoted. 

The paper is organized as follows. In the next section, we will first de- 
scribe evolutionary prisoner's dilemma game and the adjusted definition of 
fitness. Subsequently, the main simulation results and discussions will be 
presented in Section 3. And the conclusion is summarized in the last section. 



2. The Model 

For simplicity, but without loss of generality, we consider an evolutionary 
prisoner's dilemma game with the rescaled payoff matrix: T = b, R = 1 and 
P = S = 0, where the parameter 6 (1 < 6 < 2) represents the temptation 
to choose defection. Initially, cooperators and defectors are randomly dis- 
tributed on a square L x L square lattice with equal probability. At each 
time step, player i firstly acquires its payoff Pi by playing game with its di- 
rect neighbors. Next, the payoffs Pj of all the neighbors of player i can be 
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Figure 1: {color online) Characteristic snapshots of cooperators (red) and defectors (gray) 
for different values of the parameter u. From (a) to (d) u — 0, 0.2, 0.4 and 0.6, respectively. 
Depicted results were obtained for b — 1.20 and if — 0.1 on a 100 x 100 square lattice. 



obtained by means of the same way as player i, and the environment could 
be characterized by the average value of all neighbors' payoffs Pj , that is, 

P=^^, (1) 

Ki 

where the sum runs over all the neighbors of player i, and ki denotes 
the neighbor number of player i. Then we evaluate the fitness of player i 
according to the following expression: 



fi 



'1 



X P, • + M X P, 



(2) 



where < u < 1 is a tunable parameter. Obviously, when u = 0, 
the fitness mathematically equals the accumulated payoff Pj, which does not 
consider the influence of environment and goes back to the traditional version 



19l |24| . |32| . Interestingly, as u increases, the environment will hold larger 
proportion. After each step of the game, players synchronously update their 
strategies. When player i is to update its strategy, it will randomly select 
a neighbor j and adopt its strategy with a probability W(^i^j) depending on 
the fitness difference, namely 



1 



l + expU-f,)/Ky 

where < K < +oo characterizes the environmental noise, including ir- 
rationality and errors. The effect of noise K has been well studied in previous 



literatures [19|, |32|, |33|, |46 
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3. Simulation Results and Discussion 




Figure 2: (color online) (a):Frequency of cooperators Fc in dependence on the parameter 
b for different values of the parameter u. Note that comparing with the traditional version 
(i.e., u = 0), the cooperation can be greatly improved in our version, (b): Critical thresh- 
old values of 6 = 6c, marking extinction of cooperators, in dependence on the parameter 
u for different neighborhood sizes. Note that the value of be will monotonously increase 
with the the increase of u, irrespective of the neighborhood size. Due to the scale of the 
parameter b{l < b < 2), 6c > 2 is canceled. Depicted results were obtained for K — 0.1. 

Simulations below were carried out on a 100 x 100 square lattice. The key 
quantity frequency of cooperators Fc was determined within 10^ full Monte 
Carlo steps after sufficiently long transients were discarded. Moreover, each 
data were averaged over up to 20 independent runs for each set of parameter 
values in order to assure suitable accuracy. 

We start by visually inspecting the spatial patterns formed by coopera- 
tors and defectors for different values of the adjusting parameter u. Figure 
1 shows the results obtained by 6 = 1.20 and K = 0.1. For u = (Fig. 1(a)), 
the model returns the traditional model and the system falls into the pure- 
defector state when b is not very large; for /i = 0.2 (Fig.l(b)), a small fraction 
of cooperators can sustain by forming the small yet compact clusters to re- 
sist the invasion of defectors; for u = 0.4 (Fig. 1(c)), cooperators outperform 
defectors and they can form strong and large clusters to survive and prevail; 
for u = 0.6 (Fig. 1(d)), the system will be entirely dominated by cooperation 
strategy. These results reveal that considering the environment, i.e., inte- 
grating neighborhoods in the evaluation of fitness, can greatly enhance the 
evolution of cooperation. 
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Figure 3: (color online) Three typical sub-patterns in the spatial game: (a) a defector 
surrounded by cooperators; (b) a cooperator surrounded by defectors; (c) cooperators and 
defectors are evenly mixed. 



To quantify the effect of u more precisely, we examine the relationship 
between the frequency of cooperators Fc and the temptation to defect b for 
different values of u. As is shown in Fig.2(a), cooperation monotonously de- 
creases with the increase of b no matter under what value of u. Interestingly, 
it is evident that u plays a crucial role in the evolution of cooperation: the 
larger the value of u, the higher the cooperation level. In particular, when u 
is considerably large {u > 0.8), the defectors still go extinct even if defectors 
possess great advantage over cooperators (6 = 2.0), which indicates that the 
effect of u [i.e., integrating neighborhoods in the evaluation of fitness) on 
facilitating cooperation is very conspicuous. Moreover, it is also worth not- 
ing how the critical value be, marking the extinction of cooperators, varies 
in dependence on the parameter u for different neighborhood sizes. Fig. 2(b) 
demonstrates clearly the existence of an optimal neighborhood size when the 
value of u is small, that is, cooperation can be better promoted in an interme- 
diate neighborhood size, which is in agreement with the result of traditional 



version 



44l |. Whereas, as the value of u increasing this trend will change, and 
small neighborhood size will be more beneficial for the evolution of coopera- 
tion. More importantly, we can observe that although the survival space for 
cooperators is becoming smaller and smaller, which could be predicted from 



the mean- field approximation j3J, |45| , the threshold value of cooperators be 



will monotonously enhance with the increasing of u, irrespective of the neigh- 
borhood sizes. This suggests that such a facilitation effect on cooperation is 
robust to different neighborhood sizes. 
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Figure 4: {color online) The transition possibility between the circled cooperator and 
defector in Fig. 3. (a) corresponds to the pattern of Fig. 3(a); (b) corresponds to the pattern 
of Fig. 3(b); (c) corresponds to the pattern of Fig. 3(c). Depicted results were obtained for 
if = 0.1. 



Next we will analyze the underlying mechanism for this kind of promo- 
tion effect. Figure 3 shows three typical sub-patterns on square lattices, 
pattern (a): a defector is surrounded by cooperators; pattern (b): a cooper- 
ator encircled by defectors; pattern (c): cooperators and defectors are evenly 
mixed. Here we will focus on the strategy transition between the circled red 
cooperator and the circled blue defector. Before the formal analysis, it is in- 
dispensable to predefine some useful parameters. We assume that 
(/d5 foy Id) s^re the fitness of the circled cooperator (defector) in pattern (a), 
pattern (b), pattern (c), respectively; A^, A;,, Ac are the fitness difference of 
the circled cooperator and the circled defector in pattern (a), pattern (b), 
pattern (c), respectively; and W^^^,, W^^j^, W^^j^ {W^^c^ ^d^c^ ^d^c ) 
are the transition possibility of the circled cooperator to become defector (the 
circled defector to become cooperator) in pattern (a), pattern (b), pattern 
(c), respectively. According to Eq.(2), we have: 



A, 


= fi 


-fc 


= (46 


- 3) + (3.75 - 56)m 






= fh 


-fh 


= 6(1 


-2m) 


(4) 


Ac 


= fh 


-fc 


= 2(6 


-l)(l-n). 





The fitness difference of cooperators and defectors is crucial for the evolu- 
tion of cooperation. Several previous works have shown that the emergence of 
cooperation on lattices is often induced by the formation of cooperator clus- 
ters, where cooperators can obtain higher payoff (fitness) to protect them- 
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selves against the invasion of defectors. From Eq.(4), we can find A^, Af,, A^ 
monotonously decrease with the increase of u, which indicates large value 
of u can weaken the advantage of defectors, and will be greatly beneficial 
for the evolution of cooperation. Additionally, Figure 4 clearly demonstrates 
the transition possibility of the circled cooperator and circled defector when 
varying value of b for three sub-patterns. One can observe that, although 
the cross points of Wn^c and Wc^d are different, the result is consistent 
that as u increases, a defector is more likely to turn into a cooperator yet a 
cooperator is harder to become a defector. 

1.3 
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0.9 
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Figure 5: {color online)F\x\\ b — K phase diagram for u — 0.2 (a) and u — 0.4 (b), obtained 
via Monte Carlo simulations of the prisoner's dilemma game on the square lattice. The 
blue and red lines mark the border between stationary pure C and D phases and the mixed 
C+D phase, respectively. Resembling with previous works i^, 12], it can be observed that 
for u = 0.2 (a) there exists an intermediate uncertainty in the strategy adoption process 
(an intermediate value of K) for which the survivability of cooperators is maximal, i.e., Fq 
is maximal. However, the borderline separating the pure C and the mixed C+D phase for 
the u = 0.4 case (b) exhibits a monotonically decreasing curve, the D C+D transition 
is qualitatively different. 



Furthermore, we will examine how the parameter u affect the evolution 
of cooperation under different levels of uncertainty. The later can be tuned 
via K (see Eq. (3)), which acts as a temperature parameter in the employed 



Fermi strategy adoption function llj. In the case of i^' — >• 0, the strategy of 



selected neighbor is always adopted provided that its fitness is higher. While 
in the limit K — > 00, all information is lost, and switching to neighbor's 
strategy is like tossing a coin. For accurately solving the problem, we obtain 
the full b — K phase diagrams for some characteristic values of u on the square 
lattice. 
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The phase diagram illustrated in Fig. 5 (a) is well-known, and implies the 
existence of an optimal level of uncertainty for the evolution cooperation, as 



was previously reported in [20|, 122[. In particular, note the the D -H- C+D 
transition line is bell shaped, indicating that K ^ 0.3 is the optimal temper- 
ature where cooperators are able to survive at the highest value of b. This 
phenomenon as the evolution resonance can only be observed on the inter- 



action topologies lacking the overlapping triangles |19l| . On the other hand, 
comparing to the traditional phase diagram |2ol . [iii 32|. i.e. u = 0, the 
survival space of mixed strategies C-l-D is enhanced greatly, which will be 
helpful for the emergence of cooperation. Interestingly, the increasing of u 
completely eradicates (as do interaction networks incorporating overlapping 
triangles) the existence of an optimal K, as presented in Fig. 5(b). The bell 
shaped D o C+D transition line gives way to a monotonously decreasing 
line, indicating the enhancement of uncertainty level will directly accelerate 
the dying out of cooperators. This quantitative change in the phase diagram 
implies that increasing the tunable parameter u or the proportion of environ- 
ment in the individual fitness will affectively alter the interaction networks. 
Though the square lattice evidently lacks the overlapping triangles, which 
guarantees the existence of an optimal K for the small values of u, consid- 
ering neighborhoods in the fitness seems to alter the likelihood and enables 
the linkage of essentially disconnected triplets realized. A similar report was 
recently investigated in public goods games, in which the joint membership 
in large groups was also found to alter the effective interaction network and 



thus the impact of uncertainly on the evolution of cooperation |46 



Lastly, it remains of interest to explore the generalization of cooperation 
promotion for such a new mechanism, we investigate the systematic cooper- 
ative behaviors under different complex networks and different game models 
(Fig. 6). Similarly as Figure 2, it can be observed that the cooperation level 
can be greatly promoted with the increzse of u for prisoner's dilemma game 
on the small world network and the random regular graph. At the same time, 
it shows that the evolution of cooperation can be effectively guaranteed for 
snowdrift game and pubhc goods game on square lattices. The result is spe- 
cially inspiring in snowdrift game, since cooperation is usually thought to be 



inhibited by the spatial structure [36 
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Figure 6: (color online) Frequency of cooperators Fc in dependence on the parameter b 
(or r) for different values of tlie parameter u for different complex networks and different 
game models: (a) prisoner's dilemma game on the Watts-Strogatz small world network 
(SW) with the fraction of rewired links equalling 0.1; (b) prisoner's dilemma game on the 
random regular graph (RRG); (c) snowdrift game on the regular lattice; (d) public goods 
games on the regular lattice. Note that these results are in qualitative agreement with 
those obtained in Fig. 2(a) in that the larger value of u greatly facilitates cooperation. 
Depicted results were obtained for X = 0.1. 
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4. Conclusion 

In summary, we have investigated the effect of redefined fitness on the 
cooperative behaviors whin the framework of spatial evolutionary prisoner's 
dilemma game. In the model, the fitness of an individual is evaluated by the 
combination of its current payoff and environment (the average value of all 
its neighbors' payoffs). It has been found that integrating neighborhoods in 
the evaluation of fitness can promote a remarkably high cooperation level, 
especially when it holds larger proportion in the fitness. Via exploring some 
typical sub-patterns in spatial games, we have demonstrated that cooperators 
considering the influence of environment can better resist the invasion of 
defectors. 

Further interesting is the fact the increase of the parameter u, marking 
more proportion of environment, has altered the effective interaction net- 
work, while the similar phenomenon can be observed for public goods games 
in [4q| . This effective transition of topology structure will provide more ben- 
eficial condition to cooperators. To prove the generality of the promotion 
effect, we have also examined the cooperative behaviors for different neigh- 
borhood sizes, different topology structures and different game models. It is 
instructive that the similar results can be observed. Our work reveals that 
considering environment i.e., integrating neighborhoods in the evaluation of 
fitness, plays an important role in the evolution of cooperation, and thus it 
may shed a light on understanding the emergence of cooperative behaviors 
in natural and social systems. 
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